Computing systems today perform extensive amount of calculations. Developers of applications for computing systems often consider the finite precision of floating point hardware in processor cores and the resulting potential for small inaccuracies to snowball into larger, often glaring, inaccuracies over the course of a long sequence of computations.
Floating point hardware, such as a floating point unit (FPU), is a part of a computer system that is specifically designed to carry out operations on floating point numbers. Floating point refers to a representation of a number where a radix point (decimal point, or, more commonly in computers, binary point) can “float”; that is, can be placed anywhere relative to the significant digits of the number. This position is indicated separately in the internal representation, and floating-point representation can thus be thought of as a computer realization of scientific notation. Typical operations performed by floating point hardware on floating point numbers are addition, subtraction, multiplication, division, and square root. In most general purpose computer architectures, one or more FPUs are integrated with the processor.
Inaccuracies in floating point hardware operations can arise due to finite precision. The floating point unit operates to represent an infinite number of floating point values with a finite number of bits. There are some ways in which computations can become numerically inaccurate in floating point hardware. For example, it may happen due to rounding of results or due to binary/decimal conversion.
Multiple solutions have been utilized to provide numerically accurate floating point operations. One approach includes writing software with accuracy-aware algorithms that explicitly consider the finite precision of each computation. Numerical analysis is an established field, and programmers can write software that tolerates limited precision and provides the desired accuracy. The drawback to accuracy-aware code is that, compared to accuracy-unaware code, it is more complicated and involves more work to perform a given computation, thus consuming additional time and energy of the computer system.
Another approach for achieving numerical accuracy may be to use simple, accuracy-unaware algorithms, but with the greatest precision possible in the hardware. For example, this approach may replace each 32-bit single-precision “float” with a 64-bit double-precision “double.” In many situations, this approach provides the target accuracy, yet often incurs extra work (energy and latency) associated with the computation and memory accesses, with little advantage obtained over the less-precise hardware implementation.
Furthermore, both of the approaches mentioned above, namely accuracy-aware algorithms and accuracy-unaware algorithms, result in storing large precision numbers in memory. This increases total power consumption of the computer system because additional memory access power is utilized for such storage.